Re: [Myexperiment-discuss] myExperiment policy about negative comments

[Paolo Missier]

Marco

this is a fine line. Anonymous peer reviews as we know them are 
degenerating to unacceptably poor quality levels even for major 
conferences (but not for high-profile journals, where editorial control 
is much tighter), at least in CS. Rebuttals as they are defined now are 
useless. It's just so easy for an anonymous reviewer who may not be top 
notch to avoid committing to a definite judgment without paying any 
price, and because it is increasingly difficult to find good competent 
reviewers for specialistic papers, at best you end up with many shy 
reviews that are not informative enough.

I believe "open science" stands a chance to steer away from this mode 
of operation and encourage "open" reviews of one another's _information 
products_. Note that this should go well beyond papers and publications, 
because now you are entitled to annotate any artifact, and in fact 
spotting a bug in a workflow is something you _do_ want to take credit for.
At the same time, this calls for self-discipline both in the content 
and the tone of the notes. But again, virtual communities everywhere 
seem to have gone past many of the growth pains of earlier days, and 
professionals should be smart enough to realize that flame wars are a 
waste of time for everyone involved, and are not long-lived.

(just my £0.02 = $0.32 at today's rate :-( )

-Paolo

>  Hello Giovanni,
>
> You raise an important point, especially because I think myExperiment 
> needs negative (but not destructive) comments to mature.
>
> For example, my first workflow on myExperiment had serious flaws that 
> a reviewer rightly spotted when we submitted a paper about it. The 
> paper did not make it. If the workflow had had negative comments, I 
> would have submitted a better paper. Ultimately, only positive 
> comments will not help anyone. NB the review reports I got were very 
> constructive, I would have had no objection if these would have been 
> 'negative' comments on myExperiment. Which does make me think: 
> myExperiment developers, can you link comments/ratings to /versions/ 
> of workflows, because naturally I improved the workflow in response to 
> the reviews?
>
> >From what I've seen on the net in forums and shopping sites, it is 
> all about the tone of the comments. It is quite possible to bring 
> forward constructive negative comments. As myExperiment matures, I 
> would like when moderators could remove destructive comments and block 
> destructive commenters. Also, I think we should see what the new 
> generation of scientists, brought up on blog- and online shopping 
> sites, will make of myExperiment.
>
> Additionally, I've been an advocate of additional expert rating on 
> myExperiment. This is common on shopping sites, and would be 
> reminiscent of peer review. This may add credibility for some users of 
> myExperiment, but perhaps more importantly add creditability for the 
> traditional scientific community. I imagine selected experts would be 
> asked to review workflows (or packs of scientific research objects). 
> This could be anonymous, addressing the problem you raised. On a still 
> young site like myExperiment, it may also get the rating system started.
>
> As for your angry scientists scenario's: they are true, but I don't 
> think this can be solved completely. I happens in anonymous peer 
> review just the same. We generally guess who is refereeing, don't we? 
> In the user ratings everything is out in the open, so a notorious foul 
> commenter would be visible too. What about rating raters? ;-)
>
> Hope my two cents help,
> Marco.