Dear newsgroup readers,
I would like to ask for your help.
I'm a student at the Computer Science department of the Oregon
Graduate Institute. I have been following the developments in the Open
Source community for about 4 years and I've been fascinated with the
way Open Source movement works. I've been doing research in empirical
software engineering and I was wondering if the same principle that
drives the Open Software movement can work in other areas.
The main difficulty in empirical software engineering studies is the
great variability in people's performance (sometimes as much as 25:1),
which causes a lot of "noise" in analysis. Therefore studies with 30
or 40 participants are often just not powerful enough to get credible
results.
This gave me the idea of an "Open software engineering study". Using
the same principle utilized by the Open Source software, such studies
could leverage contributions of multiple participants spread all over
the world to overcome the "noise" problem and help us better
understand the effectiveness of different software engineering methods
and techniques. All the participants would have to do is connect to a
central web server and spend a little time performing some simple
software engineering tasks online. The large number of participants
would allow us to filter out the "noise" and make a justified
conclusion about which techniques work better under which
circumstances.
This kind of study has never been attempted before and some of the
professors on my committee are not convinced that such studies would
be able to attract enough volunteers to make them worthwhile.
This is where I need your help. I have put together a survey to find
out if there are enough people in the world who would like to make a
contribution to the field by participating in an internet-based
software engineering study. If you are interested in participating (or
even if you are not), please take 10 minutes out of your busy schedule
to take the survey. The results of this survey should either convince
my committee that online studies might work, or convince me that I'm
wasting my time working on them.
All people who take the survey will be entered in a drawing for
prizes.
To take the survey, point your web browsers to
http://software-engineering.cse.ogi.edu/surv/driver.cgi?R=FG
If you have any questions or comments please email me at
·····@cse.ogi.edu or call (503)748-7385. If you know someone who might
be interested in participating, please pass this information to them!
Thank you for your time,
Alex Kotov
Pacific Software Research Center
Oregon Graduate Institute of Science and Technology
On Thu, 27 May 1999 17:36:43 -0700, Alex Kotov <·····@cse.ogi.edu> wrote:
:The main difficulty in empirical software engineering studies is the
:great variability in people's performance (sometimes as much as 25:1),
:which causes a lot of "noise" in analysis. Therefore studies with 30
:or 40 participants are often just not powerful enough to get credible
:results.
It makes it more difficult, but there are design techniques for experiments
and statistical analyses that might work well.
Almost nobody in a computer science department will know about
them---you should try talking to some experimental psychologists and
biostatisticians.
This is very important, and might be crucial to your success.
You might also want to make models of what results would be and simulate them
to discern what sort of statistical results you would expect to estimate
your power, i.e. what population size do you expect to give significant
results.
:This gave me the idea of an "Open software engineering study". Using
:the same principle utilized by the Open Source software, such studies
:could leverage contributions of multiple participants spread all over
:the world to overcome the "noise" problem and help us better
:understand the effectiveness of different software engineering methods
:and techniques.
A great idea!
:This kind of study has never been attempted before and some of the
:professors on my committee are not convinced that such studies would
:be able to attract enough volunteers to make them worthwhile.
Sadly a common CS experience, that few people understand the value of
actually doing science instead of constructive mathematics. (It costs more
$).
--
* Matthew B. Kennel/Institute for Nonlinear Science, UCSD
*
* "do || !do; try: Command not found"
* /sbin/yoda --help
* ···@#!&@*#!&( Not a typewriter @!\\~~*1^#&*(@!&("
* /sbin/kosh --help
Matt Kennel wrote:
> On Thu, 27 May 1999 17:36:43 -0700, Alex Kotov <·····@cse.ogi.edu> wrote:
> :The main difficulty in empirical software engineering studies is the
> :great variability in people's performance (sometimes as much as 25:1),
> :which causes a lot of "noise" in analysis. Therefore studies with 30
> :or 40 participants are often just not powerful enough to get credible
> :results.
>
> It makes it more difficult, but there are design techniques for experiments
> and statistical analyses that might work well.
>
> Almost nobody in a computer science department will know about
> them---you should try talking to some experimental psychologists and
> biostatisticians.
>
> This is very important, and might be crucial to your success.
>
> You might also want to make models of what results would be and simulate them
> to discern what sort of statistical results you would expect to estimate
> your power, i.e. what population size do you expect to give significant
> results.
>
> :This gave me the idea of an "Open software engineering study". Using
> :the same principle utilized by the Open Source software, such studies
> :could leverage contributions of multiple participants spread all over
> :the world to overcome the "noise" problem and help us better
> :understand the effectiveness of different software engineering methods
> :and techniques.
>
> A great idea!
>
> :This kind of study has never been attempted before and some of the
> :professors on my committee are not convinced that such studies would
> :be able to attract enough volunteers to make them worthwhile.
>
> Sadly a common CS experience, that few people understand the value of
> actually doing science instead of constructive mathematics. (It costs more
> $).
Tsk, I could say some very rude things here, but I wont. Psychology of
programming is certainly a valid topic, but I would say that this kind of work
is psychology not computer science. Not that psychology is less worthwhile that
CS of course!
I think I agree with Alex's professors, these things are a good idea in
principle but most open source programmers do so for the love of hacking and I'm
not convinced that any non-obvious results could emerge from such a study.
Calum
WARNING: NOISE AHEAD
"C. Grant" <·······@hermes.cam.ac.uk> writes:
> Tsk, I could say some very rude things here, but I wont. Psychology
> of programming is certainly a valid topic, but I would say that this
> kind of work is psychology not computer science. Not that
> psychology is less worthwhile that CS of course!
I think there is an easy consideration that can be made on the
subject.
All programmers are nuts. Which makes the study of Programmers'
Psychology either trivial of extremely complex :)
Cheers
--
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - 06 68 10 03 17, fax. +39 - 06 68 80 79 26
http://www.parades.rm.cnr.it/~marcoxa