DEFSYSTEM works. You specify your source files, their dependencies,
how to generate a target from a source (for those special cases), and
from then on it will keep the system in sync.
However, it is a manual process that should be fully automated (as
much as possible). All the information is there for LISP to deduce
the dependencies and the compilation order needed to build the system.
FOOSYSTEM is an enhanced DEFSYSTEM (I don't know if FOOSYSTEM actually
exists under a different name; that's why I'm posting this article).
Here is how FOOSYSTEM works. It is given a set of LISP source files.
It scans each file, noting which macros and functions are defined in
each file, and the dependencies of each. It then has most of the
information that would have been provided to it by DEFSYSTEM, except
that it has produced it on its own. FOOSYSTEM could go further and
report functions that were referenced in the code but are not present
in core. I realize that CL compilers already do this, but this is
occuring before compilation time, at what can be termed "system
assembly time". It could then tell the programmer, "Hey, I need a
source file that defines these functions". In other words, FOOSYSTEM
is intelligent in the sense that it will attempt to get the system to
build and only prompt the programmer about the nonobvious. Contrast
this to the DEFSYSTEM of today, which has to be told _everything_.
--
Samir Barjoud
·····@mindspring.com
In article <··············@mindspring.com>, Samir Barjoud <·····@mindspring.com> wrote:
> DEFSYSTEM works. You specify your source files, their dependencies,
> how to generate a target from a source (for those special cases), and
> from then on it will keep the system in sync.
>
> However, it is a manual process that should be fully automated (as
> much as possible). All the information is there for LISP to deduce
> the dependencies and the compilation order needed to build the system.
>
> FOOSYSTEM is an enhanced DEFSYSTEM (I don't know if FOOSYSTEM actually
> exists under a different name; that's why I'm posting this article).
> Here is how FOOSYSTEM works. It is given a set of LISP source files.
> It scans each file, noting which macros and functions are defined in
> each file, and the dependencies of each.
From xref.lisp :
;;; Mon Jan 21 16:21:20 1991 by Mark Kantrowitz <·····@GLINDA.OZ.CS.CMU.EDU>
;;; xref.lisp
;;; ****************************************************************
;;; List Callers: A Static Analysis Cross Referencing Tool for Lisp
;;; ****************************************************************
;;;
;;; The List Callers system is a portable Common Lisp cross referencing
;;; utility. It grovels over a set of files and compiles a database of the
;;; locations of all references for each symbol used in the files.
;;; List Callers is similar to the Symbolics Who-Calls and the
;;; Xerox Masterscope facilities.
;;;
;;; When you change a function or variable definition, it can be useful
;;; to know its callers, in order to update each of them to the new
;;; definition. Similarly, having a graphic display of the structure
;;; (e.g., call graph) of a program can help make undocumented code more
;;; understandable. This static code analyzer facilitates both capabilities.
;;; The database compiled by xref is suitable for viewing by a graphical
;;; browser. (Note: the reference graph is not necessarily a DAG. Since many
;;; graphical browsers assume a DAG, this will lead to infinite loops.
;;; Some code which is useful in working around this problem is included,
;;; as well as a sample text-indenting outliner and an interface to Bates'
;;; PSGraph Postscript Graphing facility.)
;;;
;;; Written by Mark Kantrowitz, July 1990.
;;;
;;; Address: School of Computer Science
;;; Carnegie Mellon University
;;; Pittsburgh, PA 15213
;;;
;;; Copyright (c) 1990. All rights reserved.
;;;
;;; See general license below.
;;;
;;; ****************************************************************
;;; General License Agreement and Lack of Warranty *****************
;;; ****************************************************************
;;;
;;; This software is distributed in the hope that it will be useful (both
;;; in and of itself and as an example of lisp programming), but WITHOUT
;;; ANY WARRANTY. The author(s) do not accept responsibility to anyone for
;;; the consequences of using it or for whether it serves any particular
;;; purpose or works at all. No warranty is made about the software or its
;;; performance.
;;;
;;; Use and copying of this software and the preparation of derivative
;;; works based on this software are permitted, so long as the following
;;; conditions are met:
;;; o The copyright notice and this entire notice are included intact
;;; and prominently carried on all copies and supporting documentation.
;;; o No fees or compensation are charged for use, copies, or
;;; access to this software. You may charge a nominal
;;; distribution fee for the physical act of transferring a
;;; copy, but you may not charge for the program itself.
;;; o If you modify this software, you must cause the modified
;;; file(s) to carry prominent notices (a Change Log)
;;; describing the changes, who made the changes, and the date
;;; of those changes.
;;; o Any work distributed or published that in whole or in part
;;; contains or is a derivative of this software or any part
;;; thereof is subject to the terms of this agreement. The
;;; aggregation of another unrelated program with this software
;;; or its derivative on a volume of storage or distribution
;;; medium does not bring the other program under the scope
;;; of these terms.
;;; o Permission is granted to manufacturers and distributors of
;;; lisp compilers and interpreters to include this software
;;; with their distribution.
;;;
;;; This software is made available AS IS, and is distributed without
;;; warranty of any kind, either expressed or implied.
;;;
;;; In no event will the author(s) or their institutions be liable to you
;;; for damages, including lost profits, lost monies, or other special,
;;; incidental or consequential damages arising out of or in connection
;;; with the use or inability to use (including but not limited to loss of
;;; data or data being rendered inaccurate or losses sustained by third
;;; parties or a failure of the program to operate as documented) the
;;; program, even if you have been advised of the possibility of such
;;; damanges, or for any claim by any other party, whether in an action of
;;; contract, negligence, or other tortious action.
;;;
;;; The current version of this software and a variety of related
;;; utilities may be obtained by anonymous ftp from a.gp.cs.cmu.edu
;;; (128.2.242.7) or any other CS machine in the directory
;;; /afs/cs.cmu.edu/user/mkant/Public/Lisp-Utilities/
;;; You must cd to this directory in one fell swoop, as the CMU
;;; security mechanisms prevent access to other directories from an
;;; anonymous ftp. For users accessing the directory via an anonymous
;;; ftp mail server, the file README contains a current listing and
;;; description of the files in the directory. The file UPDATES describes
;;; recent updates to the released versions of the software in the directory.
;;; The file COPYING contains the current copy of this license agreement.
;;; Of course, if your site runs the Andrew File System and you have
;;; afs access, you can just cd to the directory and copy the files directly.
;;;
;;; Please send bug reports, comments, questions and suggestions to
;;; ·····@cs.cmu.edu. We would also appreciate receiving any changes
;;; or improvements you may make.
;;;
;;; If you wish to be added to the ············@cs.cmu.edu mailing list,
;;; send email to ····················@cs.cmu.edu with your name, email
;;; address, and affiliation. This mailing list is primarily for
;;; notification about major updates, bug fixes, and additions to the lisp
;;; utilities collection. The mailing list is intended to have low traffic.
;;;
;;; ********************************
;;; Change Log *********************
;;; ********************************
;;;
;;; 27-FEB-91 mk Added insert arg to psgraph-xref to allow the postscript
;;; graphs to be inserted in Scribe documents.
;;; 21-FEB-91 mk Added warning if not compiled.
;;; 07-FEB-91 mk Fixed bug in record-callers with regard to forms at
;;; toplevel.
;;; 21-JAN-91 mk Added file xref-test.lisp to test xref.
;;; 16-JAN-91 mk Added definition WHO-CALLS to parallel the Symbolics syntax.
;;; 16-JAN-91 mk Added macroexpansion capability to record-callers. Also
;;; added parameter *handle-macro-forms*, defaulting to T.
;;; 16-JAN-91 mk Modified print-caller-tree and related functions
;;; to allow the user to specify root nodes. If the user
;;; doesn't specify them, it will default to all root
;;; nodes, as before.
;;; 16-JAN-91 mk Added parameter *default-graphing-mode* to specify
;;; the direction of the graphing. Either :call-graph,
;;; where the children of a node are those functions called
;;; by the node, or :caller-graph where the children of a
;;; node are the callers of the node. :call-graph is the
;;; default.
;;; 16-JAN-91 mk Added parameter *indent-amount* to control the indentation
;;; in print-indented-tree.
;;; 16-JUL-90 mk Functions with argument lists of () were being ignored
;;; because of a (when form) wrapped around the body of
;;; record-callers. Then intent of (when form) was as an extra
;;; safeguard against infinite looping. This wasn't really
;;; necessary, so it has been removed.
;;; 16-JUL-90 mk PSGraph-XREF now has keyword arguments, instead of
;;; optionals.
;;; 16-JUL-90 mk Added PRINT-CLASS-HIERARCHY to use psgraph to graph the
;;; CLOS class hierarchy. This really doesn't belong here,
;;; and should be moved to psgraph.lisp as an example of how
;;; to use psgraph.
;;; 16-JUL-90 mk Fixed several caller patterns. The pattern for member
;;; had an error which caused many references to be missed.
;;; 16-JUL-90 mk Added ability to save/load processed databases.
;;; 5-JUL-91 mk Fixed warning of needing compilation to occur only when the
;;; source is loaded.
;;; ********************************
;;; To Do **************************
;;; ********************************
;;;
;;; Verify that:
;;; o null forms don't cause it to infinite loop.
;;; o nil matches against null argument lists.
;;; o declarations and doc are being ignored.
;;;
;;; Would be nice if in addition to showing callers of a function, it
;;; displayed the context of the calls to the function (e.g., the
;;; immediately surrounding form). This entails storing entries of
;;; the form (symbol context*) in the database and augmenting
;;; record-callers to keep the context around. The only drawbacks is
;;; that it would cons a fair bit. If we do this, we should store
;;; additional information as well in the database, such as the caller
;;; pattern type (e.g., variable vs. function).
;;;
;;; Write a translator from BNF (at least as much of BNF as is used
;;; in CLtL2), to the format used here.
;;;
;;; Should automatically add new patterns for new functions and macros
;;; based on their arglists. Probably requires much more than this
;;; simple code walker, so there isn't much we can do.
;;;
;;; Defmacro is a problem, because it often hides internal function
;;; calls within backquote and quote, which we normally ignore. If
;;; we redefine QUOTE's pattern so that it treats the arg like a FORM,
;;; we'll probably get them (though maybe the syntax will be mangled),
;;; but most likely a lot of spurious things as well.
;;;
;;; Define an operation for Defsystem which will run XREF-FILE on the
;;; files of the system. Or yet simpler, when XREF sees a LOAD form
;;; for which the argument is a string, tries to recursively call
;;; XREF-FILE on the specified file. Then one could just XREF-FILE
;;; the file which loads the system. (This should be a program
;;; parameter.)
;;;
;;; Have special keywords which the user may place in a file to have
;;; XREF-FILE ignore a region.
;;;
;;; Should we distinguish flet and labels from defun? I.e., note that
;;; flet's definitions are locally defined, instead of just lumping
;;; them in with regular definitions.
;;;
;;; Add patterns for series, loop macro.
;;;
;;; Need to integrate the variable reference database with the other
;;; databases, yet maintain separation. So we can distinguish all
;;; the different types of variable and function references, without
;;; multiplying databases.
;;;
;;; Would pay to comment record-callers and record-callers* in more
;;; depth.
;;;
;;; (&OPTIONAL &REST &KEY &AUX &BODY &WHOLE &ALLOW-OTHER-KEYS &ENVIRONMENT)
;;; ********************************
;;; Notes **************************
;;; ********************************
;;;
;;; XREF has been tested (successfully) in the following lisps:
;;; CMU Common Lisp (M2.9 15-Aug-90, Compiler M1.8 15-Aug-90)
;;; Macintosh Allegro Common Lisp (1.3.2)
;;; ExCL (Franz Allegro CL 3.1.12 [DEC 3100] 3/30/90)
;;; Lucid CL (Version 2.1 6-DEC-87)
;;;
;;; XREF has been tested (unsuccessfully) in the following lisps:
;;; Ibuki Common Lisp (01/01, October 15, 1987)
;;; - if interpreted, runs into stack overflow
;;; - does not compile (tried ibcl on Suns, PMAXes and RTs)
;;; seems to be due to a limitation in the c compiler.
;;;
;;; XREF needs to be tested in the following lisps:
;;; Symbolics Common Lisp (8.0)
;;; Lucid Common Lisp (3.0, 4.0)
;;; KCL (June 3, 1987 or later)
;;; AKCL (1.86, June 30, 1987 or later)
;;; TI (Release 4.1 or later)
;;; Golden Common Lisp (3.1 IBM-PC)
;;; VAXLisp (2.0, 3.1)
;;; HP Common Lisp (same as Lucid?)
;;; Procyon Common Lisp
;;; ****************************************************************
;;; Documentation **************************************************
;;; ****************************************************************
;;;
;;; XREF analyzes a user's program, determining which functions call a
;;; given function, and the location of where variables are bound/assigned
;;; and used. The user may retrieve this information for either a single
;;; symbol, or display the call graph of portions of the program
;;; (including the entire program). This allows the programmer to debug
;;; and document the program's structure.
;;;
;;; XREF is primarily intended for analyzing large programs, where it is
;;; difficult, if not impossible, for the programmer to grasp the structure
;;; of the whole program. Nothing precludes using XREF for smaller programs,
;;; where it can be useful for inspecting the relationships between pieces
;;; of the program and for documenting the program.
;;;
;;; Two aspects of the Lisp programming language greatly simplify the
;;; analysis of Lisp programs:
;;; o Lisp programs are naturally represented as data.
;;; Successive definitions from a file are easily read in
;;; as list structure.
;;; o The basic syntax of Lisp is uniform. A list program
;;; consists of a set of nested forms, where each form is
;;; a list whose car is a tag (e.g., function name) that
;;; specifies the structure of the rest of the form.
;;; Thus Lisp programs, when represented as data, can be considered to be
;;; parse trees. Given a grammar of syntax patterns for the language, XREF
;;; recursively descends the parse tree for a given definition, computing
;;; a set of relations that hold for the definition at each node in the
;;; tree. For example, one kind of relation is that the function defined
;;; by the definition calls the functions in its body. The relations are
;;; stored in a database for later examination by the user.
;;;
;;; While XREF currently only works for programs written in Lisp, it could
;;; be extended to other programming languages by writing a function to
;;; generate parse trees for definitions in that language, and a core
;;; set of patterns for the language's syntax.
;;;
;;; Since XREF normally does a static syntactic analysis of the program,
;;; it does not detect references due to the expansion of a macro definition.
;;; To do this in full generality XREF would have to have knowledge about the
;;; semantics of the program (e.g., macros which call other functions to
;;; do the expansion). This entails either modifying the compiler to
;;; record the relationships (e.g., Symbolics Who-Calls Database) or doing
;;; a walk of loaded code and macroexpanding as needed (PCL code walker).
;;; The former is not portable, while the latter requires that the code
;;; used by macros be loaded and in working order. On the other hand, then
;;; we would need no special knowledge about macros (excluding the 24 special
;;; forms of Lisp).
;;;
;;; Parameters may be set to enable macro expansion in XREF. Then XREF
;;; will expand any macros for which it does not have predefined patterns.
;;; (For example, most Lisps will implement dolist as a macro. Since XREF
;;; has a pattern defined for dolist, it will not call macroexpand-1 on
;;; a form whose car is dolist.) For this to work properly, the code must
;;; be loaded before being processed by XREF, and XREF's parameters should
;;; be set so that it processes forms in their proper packages.
;;;
;;; If macro expansion is disabled, the default rules for handling macro
;;; references may not be sufficient for some user-defined macros, because
;;; macros allow a variety of non-standard syntactic extensions to the
;;; language. In this case, the user may specify additional templates in
;;; a manner similar to that in which the core Lisp grammar was specified.
;;;
;;; ********************************
;;; User Guide *********************
;;; ********************************
;;; -----
;;; The following functions are called to cross reference the source files.
;;;
;;; XREF-FILES (&rest files) [FUNCTION]
;;; Grovels over the lisp code located in source file FILES, using
;;; xref-file.
;;;
;;; XREF-FILE (filename &optional clear-tables verbose) [Function]
;;; Cross references the function and variable calls in FILENAME by
;;; walking over the source code located in the file. Defaults type of
;;; filename to ".lisp". Chomps on the code using record-callers and
;;; record-callers*. If CLEAR-TABLES is T (the default), it clears the
;;; callers database before processing the file. Specify CLEAR-TABLES as
;;; nil to append to the database. If VERBOSE is T (the default), prints
;;; out the name of the file, one progress dot for each form processed,
;;; and the total number of forms.
;;;
;;; -----
;;; The following functions display information about the uses of the
;;; specified symbol as a function, variable, or constant.
;;;
;;; LIST-CALLERS (symbol) [FUNCTION]
;;; Lists all functions which call SYMBOL as a function (function
;;; invocation).
;;;
;;; LIST-READERS (symbol) [FUNCTION]
;;; Lists all functions which refer to SYMBOL as a variable
;;; (variable reference).
;;;
;;; LIST-SETTERS (symbol) [FUNCTION]
;;; Lists all functions which bind/set SYMBOL as a variable
;;; (variable mutation).
;;;
;;; LIST-USERS (symbol) [FUNCTION]
;;; Lists all functions which use SYMBOL as a variable or function.
;;;
;;; WHO-CALLS (symbol &optional how) [FUNCTION]
;;; Lists callers of symbol. HOW may be :function, :reader, :setter,
;;; or :variable."
;;;
;;; WHAT-FILES-CALL (symbol) [FUNCTION]
;;; Lists names of files that contain uses of SYMBOL
;;; as a function, variable, or constant.
;;;
;;; SOURCE-FILE (symbol) [FUNCTION]
;;; Lists the names of files in which SYMBOL is defined/used.
;;;
;;; LIST-CALLEES (symbol) [FUNCTION]
;;; Lists names of functions and variables called by SYMBOL.
;;;
;;; -----
;;; The following functions may be useful for viewing the database and
;;; debugging the calling patterns.
;;;
;;; *LAST-FORM* () [VARIABLE]
;;; The last form read from the file. Useful for figuring out what went
;;; wrong when xref-file drops into the debugger.
;;;
;;; *XREF-VERBOSE* t [VARIABLE]
;;; When T, xref-file(s) prints out the names of the files it looks at,
;;; progress dots, and the number of forms read.
;;;
;;; *TYPES-TO-IGNORE* (quote (:lisp :lisp2)) [VARIABLE]
;;; Default set of caller types (as specified in the patterns) to ignore
;;; in the database handling functions. :lisp is CLtL 1st edition,
;;; :lisp2 is additional patterns from CLtL 2nd edition.
;;;
;;; *HANDLE-PACKAGE-FORMS* () [VARIABLE]
;;; When non-NIL, and XREF-FILE sees a package-setting form like
;;; IN-PACKAGE, sets the current package to the specified package by
;;; evaluating the form. When done with the file, xref-file resets the
;;; package to its original value. In some of the displaying functions,
;;; when this variable is non-NIL one may specify that all symbols from a
;;; particular set of packages be ignored. This is only useful if the
;;; files use different packages with conflicting names.
;;;
;;; *HANDLE-FUNCTION-FORMS* t [VARIABLE]
;;; When T, XREF-FILE tries to be smart about forms which occur in
;;; a function position, such as lambdas and arbitrary Lisp forms.
;;; If so, it recursively calls record-callers with pattern 'FORM.
;;; If the form is a lambda, makes the caller a caller of
;;; :unnamed-lambda.
;;;
;;; *HANDLE-MACRO-FORMS* t [VARIABLE]
;;; When T, if the file was loaded before being processed by XREF, and
;;; the car of a form is a macro, it notes that the parent calls the
;;; macro, and then calls macroexpand-1 on the form.
;;;
;;; *DEFAULT-GRAPHING-MODE* :call-graph [VARIABLE]
;;; Specifies whether we graph up or down. If :call-graph, the children
;;; of a node are the functions it calls. If :caller-graph, the
;;; children of a node are the functions that call it.
;;;
;;; *INDENT-AMOUNT* 3 [VARIABLE]
;;; Number of spaces to indent successive levels in PRINT-INDENTED-TREE.
;;;
;;; DISPLAY-DATABASE (&optional database types-to-ignore) [FUNCTION]
;;; Prints out the name of each symbol and all its callers. Specify
;;; database :callers (the default) to get function call references,
;;; :file to the get files in which the symbol is called, :readers to get
;;; variable references, and :setters to get variable binding and
;;; assignments. Ignores functions of types listed in types-to-ignore.
;;;
;;; PRINT-CALLER-TREES (&key (mode *default-graphing-mode*) [FUNCTION]
;;; (types-to-ignore *types-to-ignore*)
;;; compact root-nodes)
;;; Prints the calling trees (which may actually be a full graph and not
;;; necessarily a DAG) as indented text trees using
;;; PRINT-INDENTED-TREE. MODE is :call-graph for trees where the children
;;; of a node are the functions called by the node, or :caller-graph for
;;; trees where the children of a node are the functions the node calls.
;;; TYPES-TO-IGNORE is a list of funcall types (as specified in the
;;; patterns) to ignore in printing out the database. For example,
;;; '(:lisp) would ignore all calls to common lisp functions. COMPACT is
;;; a flag to tell the program to try to compact the trees a bit by not
;;; printing trees if they have already been seen. ROOT-NODES is a list
;;; of root nodes of trees to display. If ROOT-NODES is nil, tries to
;;; find all root nodes in the database.
;;;
;;; MAKE-CALLER-TREE (&optional (mode *default-graphing-mode*) [FUNCTION]
;;; (types-to-ignore *types-to-ignore*)
;;; compact)
;;; Outputs list structure of a tree which roughly represents the
;;; possibly cyclical structure of the caller database.
;;; If mode is :call-graph, the children of a node are the functions
;;; it calls. If mode is :caller-graph, the children of a node are the
;;; functions that call it.
;;; If compact is T, tries to eliminate the already-seen nodes, so
;;; that the graph for a node is printed at most once. Otherwise it will
;;; duplicate the node's tree (except for cycles). This is usefull
;;; because the call tree is actually a directed graph, so we can either
;;; duplicate references or display only the first one.
;;;
;;; DETERMINE-FILE-DEPENDENCIES (&optional database) [FUNCTION]
;;; Makes a hash table of file dependencies for the references listed in
;;; DATABASE. This function may be useful for automatically resolving
;;; file references for automatic creation of a system definition
;;; (defsystem).
;;;
;;; PRINT-FILE-DEPENDENCIES (&optional database) [FUNCTION]
;;; Prints a list of file dependencies for the references listed in
;;; DATABASE. This function may be useful for automatically computing
;;; file loading constraints for a system definition tool.
;;;
;;; WRITE-CALLERS-DATABASE-TO-FILE (filename) [FUNCTION]
;;; Saves the contents of the current callers database to a file. This
;;; file can be loaded to restore the previous contents of the
;;; database. (For large systems it can take a long time to crunch
;;; through the code, so this can save some time.)
;;;
;;; -----
;;; The following macros define new function and macro call patterns.
;;; They may be used to extend the static analysis tool to handle
;;; new def forms, extensions to Common Lisp, and program defs.
;;;
;;; DEFINE-PATTERN-SUBSTITUTION (name pattern) [MACRO]
;;; Defines NAME to be equivalent to the specified pattern. Useful for
;;; making patterns more readable. For example, the LAMBDA-LIST is
;;; defined as a pattern substitution, making the definition of the
;;; DEFUN caller-pattern simpler.
;;;
;;; DEFINE-CALLER-PATTERN (name pattern &optional caller-type) [MACRO]
;;; Defines NAME as a function/macro call with argument structure
;;; described by PATTERN. CALLER-TYPE, if specified, assigns a type to
;;; the pattern, which may be used to exclude references to NAME while
;;; viewing the database. For example, all the Common Lisp definitions
;;; have a caller-type of :lisp or :lisp2, so that you can exclude
;;; references to common lisp functions from the calling tree.
;;;
;;; DEFINE-VARIABLE-PATTERN (name &optional caller-type) [MACRO]
;;; Defines NAME as a variable reference of type CALLER-TYPE. This is
;;; mainly used to establish the caller-type of the variable.
;;;
;;; DEFINE-CALLER-PATTERN-SYNONYMS (source destinations) [MACRO]
;;; For defining function caller pattern syntax synonyms. For each name
;;; in DESTINATIONS, defines its pattern as a copy of the definition
;;; of SOURCE. Allows a large number of identical patterns to be defined
;;; simultaneously. Must occur after the SOURCE has been defined.
;;;
;;; -----
;;; This system includes pattern definitions for the latest
;;; common lisp specification, as published in Guy Steele,
;;; Common Lisp: The Language, 2nd Edition.
;;;
;;; Patterns may be either structures to match, or a predicate
;;; like symbolp/numberp/stringp. The pattern specification language
;;; is similar to the notation used in CLtL2, but in a more lisp-like
;;; form:
;;; (:eq name) The form element must be eq to the symbol NAME.
;;; (:test test) TEST must be true when applied to the form element.
;;; (:typep type) The form element must be of type TYPE.
;;; (:or pat1 pat2 ...) Tries each of the patterns in left-to-right order,
;;; until one succeeds.
;;; Equivalent to { pat1 | pat2 | ... }
;;; (:rest pattern) The remaining form elements are grouped into a
;;; list which is matched against PATTERN.
;;; (:optional pat1 ...) The patterns may optionally match against the
;;; form element.
;;; Equivalent to [ pat1 ... ].
;;; (:star pat1 ...) The patterns may match against the patterns
;;; any number of times, including 0.
;;; Equivalent to { pat1 ... }*.
;;; (:plus pat1 ...) The patterns may match against the patterns
;;; any number of times, but at least once.
;;; Equivalent to { pat1 ... }+.
;;; &optional, &key, Similar in behavior to the corresponding
;;; &rest lambda-list keywords.
;;; FORM A random lisp form. If a cons, assumes the
;;; car is a function or macro and tries to
;;; match the args against that symbol's pattern.
;;; If a symbol, assumes it's a variable reference.
;;; :ignore Ignores the corresponding form element.
;;; NAME The corresponding form element should be
;;; the name of a new definition (e.g., the
;;; first arg in a defun pattern is NAME.
;;; FUNCTION, MACRO The corresponding form element should be
;;; a function reference not handled by FORM.
;;; Used in the definition of apply and funcall.
;;; VAR The corresponding form element should be
;;; a variable definition or mutation. Used
;;; in the definition of let, let*, etc.
;;; VARIABLE The corresponding form element should be
;;; a variable reference.
;;;
;;; In all other pattern symbols, it looks up the symbols pattern substitution
;;; and recursively matches against the pattern. Automatically destructures
;;; list structure that does not include consing dots.
;;;
;;; Among the pattern substitution names defined are:
;;; STRING, SYMBOL, NUMBER Appropriate :test patterns.
;;; LAMBDA-LIST Matches against a lambda list.
;;; BODY Matches against a function body definition.
;;; FN Matches against #'function, 'function,
;;; and lambdas. This is used in the definition
;;; of apply, funcall, and the mapping patterns.
;;; and others...
;;;
;;; Here's some sample pattern definitions:
;;; (define-caller-pattern defun
;;; (name lambda-list
;;; (:star (:or documentation-string declaration))
;;; (:star form))
;;; :lisp)
;;; (define-caller-pattern funcall (fn (:star form)) :lisp)
;;;
;;; In general, the system is intelligent enough to handle any sort of
;;; simple funcall. One only need specify the syntax for functions and
;;; macros which use optional arguments, keyword arguments, or some
;;; argument positions are special, such as in apply and funcall, or
;;; to indicate that the function is of the specified caller type.
;;;
;;;
;;; NOTES:
;;;
;;; XRef assumes syntactically correct lisp code.
;;;
;;; This is by no means perfect. For example, let and let* are treated
;;; identically, instead of differentiating between serial and parallel
;;; binding. But it's still a useful tool. It can be helpful in
;;; maintaining code, debugging problems with patch files, determining
;;; whether functions are multiply defined, and help you remember where
;;; a function is defined or called.
;;;
;;; XREF runs best when compiled.
;;; ********************************
;;; References *********************
;;; ********************************
;;;
;;; Xerox Interlisp Masterscope Program:
;;; Larry M Masinter, Global program analysis in an interactive environment
;;; PhD Thesis, Stanford University, 1980.
;;;
;;; Symbolics Who-Calls Database:
;;; User's Guide to Symbolics Computers, Volume 1, Cambridge, MA, July 1986
;;; Genera 7.0, pp 183-185.
;;;
* Rainer Joswig
| ;;; ****************************************************************
| ;;; List Callers: A Static Analysis Cross Referencing Tool for Lisp
| ;;; ****************************************************************
| ;;;
| ;;; The List Callers system is a portable Common Lisp cross referencing
| ;;; utility. It grovels over a set of files and compiles a database of the
| ;;; locations of all references for each symbol used in the files.
| ;;; List Callers is similar to the Symbolics Who-Calls and the
| ;;; Xerox Masterscope facilities.
probably quite useful, but those who have access to Allegro CL will find
a few utilities that may not be all that trivial to locate if you don't
know that they exist. at the top-level, :WHO-CALLS for functions and
:WHO-BINDS, :WHO-SETS, :WHO-REFERENCES, and :WHO-USES for variables.
they work by having the compiler record cross referencing information in
FASL files, which is electively loaded. (EXCL:EXPLAIN-COMPILER-SETTINGS)
prints a summary of the compiler settings, including controlling the
cross referencing features.
#:Erik
--
save the children: just say NO to sex with pro-lifers
Erik Naggum wrote:
>
> * Rainer Joswig
> | ;;; ****************************************************************
> | ;;; List Callers: A Static Analysis Cross Referencing Tool for Lisp
> | ;;; ****************************************************************
...
> Probably quite useful, but those who have access to Allegro CL will find
> a few utilities that may not be all that trivial to locate if you don't
> know that they exist.
...
LW has the same facilities such as list callers, list callees and go to
source. Maybe there are undocumented functions that access the internal
dependence databases in both implementations and others.
Robert
Robert Monfera <·······@fisec.com> writes:
> Erik Naggum wrote:
> >
> > * Rainer Joswig
> > | ;;; ****************************************************************
> > | ;;; List Callers: A Static Analysis Cross Referencing Tool for Lisp
> > | ;;; ****************************************************************
> ...
> > Probably quite useful, but those who have access to Allegro CL will find
> > a few utilities that may not be all that trivial to locate if you don't
> > know that they exist.
> ...
> LW has the same facilities such as list callers, list callees and go to
> source. Maybe there are undocumented functions that access the internal
> dependence databases in both implementations and others.
... the advantage of XREF (Copyright notice apart) is that it could be
easily ported to ACL, LW, CMUCL, GCL, MCL, GENERA and so on. Thus
providing a "Common" interface, which is implementation independent.
Cheers
--
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - 06 68 10 03 17, fax. +39 - 06 68 80 79 26
http://www.parades.rm.cnr.it/~marcoxa
Marco Antoniotti wrote:
...
> the advantage of XREF (Copyright notice apart) is that it could be
> easily ported to ACL, LW, CMUCL, GCL, MCL, GENERA and so on. Thus
> providing a "Common" interface, which is implementation independent.
Rainer Joswig wrote:
...
> Just to make the difference clear - the above mentioned tool
> tries to work over source files.
>
> Most systems vendors have are collecting information using
> the compiler.
Excellent points. (I just extended on a post about ACL facilities - I
believe those are also based on compilation and loading. Originally my
mail had a sentence saying 'Of course, it may have little to do with
what Samir asked', but the thread was already diverted, so I elided it.)
Actually some of the referencing functionality works in LW just by
having a source file open, for example listing definitions and jumping
to source from it.
Does someone know how well XREF works on today's compilers and files?
(By the way, thanks to Raymond and others for maintaining the SERIES
package - pity it did not make it into the standard.)
Robert
In article <·················@fisec.com>, ·······@fisec.com wrote:
> Erik Naggum wrote:
> >
> > * Rainer Joswig
> > | ;;; ****************************************************************
> > | ;;; List Callers: A Static Analysis Cross Referencing Tool for Lisp
> > | ;;; ****************************************************************
> ...
> > Probably quite useful, but those who have access to Allegro CL will find
> > a few utilities that may not be all that trivial to locate if you don't
> > know that they exist.
> ...
> LW has the same facilities such as list callers, list callees and go to
> source. Maybe there are undocumented functions that access the internal
> dependence databases in both implementations and others.
>
> Robert
Just to make the difference clear - the above mentioned tool
tries to work over source files.
Most systems vendors have are collecting information using
the compiler.
It's a nice idea, and I have no problem with the concept of having a
defsystem form generated automatically using known
cross-reference-implementation (generated by either an
implementation-specific compiler or a portable xref tool).
However, there can be bootstrapping problems.
Because Lisp s-expression semantics are defined independently of how
they are read, you will have to consider how you will preserve the fact
that the reading of some particular defXXX form involved the use of a
reader macro.
More generally, there are some dependencies that are read-time, some
that are compile-time, and some that are run-time. Handling this
properly in the most general case requires that you essentially perform
minimal-file compilation (as defined in the spec) of each file. Now
suppose you do this for each random file in a set in some random order,
in order to learn their dependencies. Now suppose that because this
chosen order is wrong, you get an error (during read-time, or during
compile-time). How do you know that the error is due to bad file order
and not something else? Even if you allow yourself to omnisciently
define that any errors are due to processing order, how do you then fix
the problem? Do you just abort back to the begining? Begining of what?
The processing of the file that had the error? The processing of all
files? How do you unwind any side-effects that may have been introduced
by compile-time processing. For example, unsetting a defvar.
And, of course, these issues get far more complicated if your
compile-and-load process requires that definitions change during the
course of things. For example, suppose that for some complicated system
(like the PCL implementation of CLOS), you begin with some simple
definitions for bootstrapping that later get replaced by the "real"
definitions.
I guess what I'm saying is that if you want to define a very specific
set of circumstances AS A CONVENIENT HACK, in which you can hav xref or
something process some files and create a system definition file for
you, that's fine --- as long as you don't expect it to do too much. I
personally don't see how to make a general purpose, portable tool that
would REPLACE the usual defsystem tools.
Samir Barjoud wrote:
>
> DEFSYSTEM works. You specify your source files, their dependencies,
> how to generate a target from a source (for those special cases), and
> from then on it will keep the system in sync.
>
> However, it is a manual process that should be fully automated (as
> much as possible). All the information is there for LISP to deduce
> the dependencies and the compilation order needed to build the system.
>
> FOOSYSTEM is an enhanced DEFSYSTEM (I don't know if FOOSYSTEM actually
> exists under a different name; that's why I'm posting this article).
> Here is how FOOSYSTEM works. It is given a set of LISP source files.
> It scans each file, noting which macros and functions are defined in
> each file, and the dependencies of each. It then has most of the
> information that would have been provided to it by DEFSYSTEM, except
> that it has produced it on its own. FOOSYSTEM could go further and
> report functions that were referenced in the code but are not present
> in core. I realize that CL compilers already do this, but this is
> occuring before compilation time, at what can be termed "system
> assembly time". It could then tell the programmer, "Hey, I need a
> source file that defines these functions". In other words, FOOSYSTEM
> is intelligent in the sense that it will attempt to get the system to
> build and only prompt the programmer about the nonobvious. Contrast
> this to the DEFSYSTEM of today, which has to be told _everything_.
>
> --
> Samir Barjoud
> ·····@mindspring.com