Re: [Qemu-devel] Can we make better use of Coverity?

[Daniel P. Berrange]

On Wed, Jan 21, 2015 at 01:47:22PM +0100, Markus Armbruster wrote:
> We're using the Coverity Scan service[*].  We've put in some effort, and
> we've gotten some mileage out of it, but I feel we could get more.
> 
> Judging from the report e-mail I have lying about, we're scanning about
> once a month on average.  These reports cuts off after 20 new defects.
> When there are more, which is common, people have to go to the web
> dashboard to see them.  When I get one with ten, I may have a look, when
> I get one "Showing 20 of 100 defect(s)", I despair of the task, and put
> it off.
> 
> I also use Coverity locally (requires a license) with a derived model
> for GLib to increase scanning power.  Since last July, the number of
> defects I get that way has increased from ~400 to ~700.  Not quite as
> bad as it sounds, because ~100 of the new ones are DEADCODE.  Still, it
> suggests we haven't made much progress in reducing the number of defects
> to a manageable level.
> 
> Some of the new defects are avoidable.  For instance, we've added 16
> MISSING_BREAK.  Probably just missing /* fall through */, but we can't
> be sure without examining each case.  Patch review fail.
> 
> At the other end of the spectrum, I see 36 new UNINIT defects.
> 
> I think we should scan much more regularly.  Once a week, full auto?

I agree that you need to scan much more regularly. Given the number of
patches QEMU merges, with only monthly scans you're creating a big job
for whoever has to deal with the monthly report because chances are
there will be alot of new stuff reported each month to wade through.

In libvirt we now have a coverity scan being run once a day, so when
we get new problems reported, the code in question is still fresh in
the mind of the reviewers & patch author. Daily scans also spread out
the workload much better. Only get a small number of new problems to
analyse a couple of times a week - never any real huge burden for the
person managing the coverity scan & more likely to get others to help
too if there's only a couple of things for them to look at instead of
a list of 700+ to wade through. I think these contribute to make it
practical for us to keep libvirt at zero coverity problems all the
time.

If you set the current 700 issues you have reported as your baseline,
then it is still practical to run coverity daily on QEMU. Just have
it report only new issues, ignoring the backlog, and ensure those all
get fixes posted the same day so the backlog doesn't grow. Deal with
the historical backlog of issues separately as time allows.

Also I'd suggest making "no new coverity issues" be a release blocker
item so people see you are taking it seriously and so be encouraged
to help out to ensure the release doesn't slip.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|