Don't Touch My Code! Examining the Effects of Ownership on Software Quality

Prior research indicates that ownership is a good thing; that concentrating knowledge in a single person makes them an expert and thus more effective. This work investigated these claims in a quantitative way, by looking at defects and the quality of code contribute by non-owners. Their initial analysis found that a high number of contributors was correlated with post-release defects, size, and churn, and that a single owner making a high percentage of commits resulted in lower defects. They then used a regression analysis to determine if this was causation as well, or if larger projects themselves led to higher defect rates. The authors argue that including "classic" code metrics (size, churn, etc.), the total number of contributors, and the number of "minor" contributors led to the best model.

The authors then tried to form a hypothesis about why this is happening. They started with the "contributor" model, where a owner of Module A tries to fix a bug in a dependent Module B and thus a) becomes a minor contributor and b) because they are not an expert, causes a bug in Module B. This proved to be useful, and they suggest scrutinizing changes by minor contributors more closely. In addition, they suggest that minor contributors don't try to directly modify a dependent module, instead contacting the owner.

This paper was very heavily driven by statistics, drawn from data on Windows Vista and Windows 7. It was very nice to see proper statistical analysis, including actual regression tests and Monte Carlo simulations to determine if a hypothesis was based in reality. While I respect their methods and I found their conclusions compelling, I was hoping for a different answer - something more controversial like "ownership is bad". I think this is from my experience Dyninst, where ownership leads to unhappy people (e.g., Bill on IAPI fixes) or graduating students taking away significant knowledge. As a caveat (mentioned by the authors) this work was performed at Microsoft which has a strong ownership policy, and may be different in an open source project where there is a much weaker ownership policy.

ReLink: Recovering Links between Bugs and Changes

As we know, nobody actually puts bug IDs in commit messages, and so the authors built a system for automatically recovering the link between a bug report and the corresponding bug fix. In a perfect world, this would be trivial because a commit message would specify a corresponding bug or feature request in a structured format. In reality, this doesn't work, and missing bug information, irregular formats, misspellings, and the like make deriving this information difficult. The authors describe some correlations between commits and bugs reported fixed, including dates, authors, and textual similarities, and note that the usefulness of these varies between projects. They propose a solution that first mines for "explicit" links that have high accuracy probability, and learning from those what correlations should be used to recover additional "missing" links in a second pass. Their evaluation shows this is a useful approach.

I honestly don't have much to say about this talk; the problem is a good one (if a little outside my day to day scope) and their approach is effective. The presentation was also somewhat hard to follow, which probably contributes to my lack of comments. Finally, in an editorial note, it started smelling like cigarette smoke during the talk which was really distracting.

How Do Fixes Become Bugs? -- A Comprehensive Characteristic Study on Incorrect Fixes in Commercial and Open Source Operating Systems

The authors begin by noting that bug fixing behavior by developers is essentially different from writing new features - there is a tight schedule and the fixer may not be familiar with the code. As a result, there is a high chance that a bugfix will in itself cause additional bugs. They performed a study over existing operating systems, which have a high level of post-release, short time turnaround bug fixing. This study included three OSS projects (OpenSolaris, Linux, FreeBSD) and one commercial (likely NetApp). Since there was not any structured data that recorded incorrect fixes, they used a search over commit logs looking for possible incorrect fixes and then manually checked each possible.

Their first result was the ratio of incorrect bug fixes - between 15% and 20%, which itself is very high. They also note that 43% of these incorrect fixes result in crashes or security problems, and that 39% of race conditions are incorrectly fixed the first time. After this, the presentation delved into a large number of particular example cases - such as a data race fix introducing deadlock, or a deadlock fix introducing another deadlock. While interesting, the litany of stupid mistakes was not as interesting as the discussion on how to detect incorrect fixes. Their analysis suggests that, again, bug fixes by people not familiar with the particular piece of code was more likely to result in an incorrect fix.

If asked to summarize the session, I would say "code ownership is good". On the other hand, empirically we've seen that this doesn't necessarily work in a research group because it results in high concentration of workload. I wonder if a similar situation happens in OSS or commercial products, where a particular owner becomes overloaded with feature and bug fix requests while owners of other modules are idle.