In article <··············@ID-125932.news.dfncis.de>,
Christopher Browne <········@acm.org> wrote:
>Following is a summary of articles spanning a 7 day period,
>beginning at 20 Oct 2002 11:56:53 GMT and ending at
>27 Oct 2002 02:30:55 GMT.
[snip]
The results appear to be an almost perfect object lesson in why
statistical code metrics are useless without a deeper understanding of
what's going on. Both the tops and bottoms of the volume, "original
content" and thread rankings contain solid representations from the most
and least informative posters and threads of the past week.
At the level these statistics capture, there appears to be little
difference between succinct explanations and one-liners, or between
detailed technical responses and rants, and a complex discussion that
requires keeping a lot of context around looks very much like a
pedantic-mode exchange of flames. Beats KLOC and function points all
hollow though.
I'm not immediately sure how deep a parse you would have to do to make
completely reliable distinctions for this kind of thing.
paul
Paul Wallich <··@panix.com> wrote:
> At the level these statistics capture, there appears to be little
> difference between succinct explanations and one-liners, or between
> detailed technical responses and rants, and a complex discussion that
> requires keeping a lot of context around looks very much like a
> pedantic-mode exchange of flames. Beats KLOC and function points all
> hollow though.
> I'm not immediately sure how deep a parse you would have to do to make
> completely reliable distinctions for this kind of thing.
I'd be really interested to see what a naive bayesian approach would do
after I'd developed a database of a few thousand articles or so.
I'm thinking 3 categories: good, off-topic but interesting, crap.
The really interesting question is whether it could distinguish good
from bad in similar content style. e.g. distinguishing witty flamers
from whining teenagers.
On the backburner of "things I will do in my copious free time" is a
newsreader that uses this approach to write a 'probfile', as opposed to
scorefile or kill/tagfile. The basic idea is to have a way to tell the
program "This article was sorted incorrectly -- it should have been
here:" to update the database, then let it just work on the fly.
Michael
* Paul Wallich
| The results appear to be an almost perfect object lesson in why
| statistical code metrics are useless without a deeper understanding of
| what's going on. Both the tops and bottoms of the volume, "original
| content" and thread rankings contain solid representations from the most
| and least informative posters and threads of the past week.
Well, what are the metrics you have used to determine your conclusions?
| I'm not immediately sure how deep a parse you would have to do to make
| completely reliable distinctions for this kind of thing.
Readers would have to rate news articles. For the past few months, I
have been working on a system to do this with the Norwegian newsgroup
hierarchy. I may decide to repeat the experiment with other newsgroups.
--
Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
On 27 Oct 2002 23:37:12 +0000, Erik Naggum <····@naggum.no> said:
[...]
EN> Readers would have to rate news articles.
Human readers, rather than news readers, I suppose?
---Vassil.
--
For an M-person job assigned to an N-person team, only rarely M=N.
On 27 Oct 2002 21:19:02 -0500, Vassil Nikolov <········@poboxes.com> said:
On 27 Oct 2002 23:37:12 +0000, Erik Naggum <····@naggum.no> said:
[...]
EN> Readers would have to rate news articles.
VN> Human readers, rather than news readers, I suppose?
Actually, I didn't mean that to sound sarcastic. I was just
thinking that with all those developments in AI I haven't followed,
I couldn't be sure what a news reader might be able to do...
---Vassil.
--
For an M-person job assigned to an N-person team, only rarely M=N.
* Vassil Nikolov
| Human readers, rather than news readers, I suppose?
Well, I meant human, but the support for rating has to exist in both the
client and the server software.
--
Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
In article <················@naggum.no>, Erik Naggum <····@naggum.no>
wrote:
>* Paul Wallich
>| The results appear to be an almost perfect object lesson in why
>| statistical code metrics are useless without a deeper understanding of
>| what's going on. Both the tops and bottoms of the volume, "original
>| content" and thread rankings contain solid representations from the most
>| and least informative posters and threads of the past week.
>
> Well, what are the metrics you have used to determine your conclusions?
Purely subjective, based on five years or so of regular reading and name
recognition, with a sense of what posts are interesting or informative
to me and what posts appear ditto to others. I expect that someone else
would have different personal metrics, but think that most of them would
show a similar spread with respect to the stats given.
>| I'm not immediately sure how deep a parse you would have to do to make
>| completely reliable distinctions for this kind of thing.
>
> Readers would have to rate news articles. For the past few months, I
> have been working on a system to do this with the Norwegian newsgroup
> hierarchy. I may decide to repeat the experiment with other newsgroups.
Does such a system integrate reasonably with common newsreaders?
(On reflection, I think that one could probably distinguish between good
and bad in threads consisting mostly of long posts with low "original
content" by looking at posting interval and total thread length. Longer
intervals and shorter ultimate length for "useful" threads because of
the time and effort required for cogent replies, but unfortunately the
measure would be mostly retrospective.)
paul
Centuries ago, Nostradamus foresaw when Paul Wallich <··@panix.com> would write:
> In article <················@naggum.no>, Erik Naggum <····@naggum.no>
> wrote:
>>* Paul Wallich
>>| The results appear to be an almost perfect object lesson in why
>>| statistical code metrics are useless without a deeper
>>| understanding of what's going on. Both the tops and bottoms of
>>| the volume, "original content" and thread rankings contain solid
>>| representations from the most and least informative posters and
>>| threads of the past week.
>>
>> Well, what are the metrics you have used to determine your
>> conclusions?
>
> Purely subjective, based on five years or so of regular reading and
> name recognition, with a sense of what posts are interesting or
> informative to me and what posts appear ditto to others. I expect
> that someone else would have different personal metrics, but think
> that most of them would show a similar spread with respect to the
> stats given.
>
>>| I'm not immediately sure how deep a parse you would have to do to
>>| make completely reliable distinctions for this kind of thing.
>>
>> Readers would have to rate news articles. For the past few
>> months, I have been working on a system to do this with the
>> Norwegian newsgroup hierarchy. I may decide to repeat the
>> experiment with other newsgroups.
>
> Does such a system integrate reasonably with common newsreaders?
>
> (On reflection, I think that one could probably distinguish between
> good and bad in threads consisting mostly of long posts with low
> "original content" by looking at posting interval and total thread
> length. Longer intervals and shorter ultimate length for "useful"
> threads because of the time and effort required for cogent replies,
> but unfortunately the measure would be mostly retrospective.)
<http://quimby.gnus.org/gnus/manual/gnus_237.html>
"GroupLens (http://www.cs.umn.edu/Research/GroupLens/) is a
collaborative filtering system that helps you work together with other
people to find the quality news articles out of the huge volume of
news articles generated every day.
To accomplish this the GroupLens system combines your opinions about
articles you have already read with the opinions of others who have
done likewise and gives you a personalized prediction for each unread
news article. Think of GroupLens as a matchmaker. GroupLens watches
how you rate articles, and finds other people that rate articles the
same way. Once it has found some people you agree with it tells you,
in the form of a prediction, what they thought of the article. You can
use this prediction to help you decide whether or not you want to read
the article.
NOTE: Unfortunately the GroupLens system seems to have shut down, so
this section is mostly of historical interest."
You could presumably build a protocol to share Gnus "score" files with
others with similar interests which could also help.
A third approach would be to use something like Paul Graham's
statistical filtering scheme or something more sophisticated such as
IFile to filter between "good" and "bad", perhaps sharing a corpus of
"good" and "bad" material with others.
I think the ideal way of handling this would probably involve using a
GroupLens-like approach to allow people to share "scoring" information
on articles which would be used to define allocations of messages to
corpuses.
Those allocations would then be used to do IFile-like evaluations of
messages which would mean that /everyone/ would get improved scoring.
When articles were scored wrongly, the feedback would be used to
improve the corpus...
Note that the statistics are intended as much for amusement purposes
as for any serious analysis. I certainly agree that the value is
pretty dubious; you take them seriously at your own risk...
--
(concatenate 'string "cbbrowne" ·@cbbrowne.com")
http://www3.sympatico.ca/cbbrowne/internet.html
"... While programs written for Sun machines won't run unmodified on
Intel-based computers, Sun said the two packages will be completely
compatible and that software companies can convert a program from one
system to the other through a fairly straightforward and automated
process known as ``recompiling.''" -- San Jose Mercury News