Opening the source for research software is absolutely vital to the concept of reproduceability. However, this fact of the level of programming training for most scientists is a major issue. A lot of novice programmers tend to fall into a trap of "it runs without error, it must be right." Even expert programmers struggle with verifying that their results are correct; technically, program verification is a mathematical impossibility. So it's a daunting task to start with, reproducing results of software-based research.
This is only compounded by the fact that reading source code sucks. Source code is an end result of multiple processes that occur in feedback loops. With just the source code, you never see how the code got that way. It's like showing someone a maze with the start and end points marked but the middle of the map blocked out.
Different programmer's conceptions of what constitutes good code varies widely. One man's golden code is another's garbage. Just because the source code is available doesn't mean anyone is going to understand it or be able to work with it effectively.
Compounding this all is the fact that few people are going to want to read the source code. Analyzing source code is dull work, maybe the worst job a programmer can take while still doing programming. Most programmers are far happier to discard old code and start from scratch. This is often a bad idea and doesn't lead to a better product, but at least you don't want to kill yourself while you're doing it.
When it comes to reproducing algorithmic results, I would prefer having a description of the algorithm, a set of inputs, and a set of outputs. I would then write the actual code myself and see if I get the same results. This, I think, is much closer to the concept of reproducing lab results in the physical sciences. You wouldn't use the same exact particle accelerators if you were verifying the results from a paper on nuclear physics. I'm afraid having access to the raw source code will be used as a crutch where logic errors are missed from reusing portions of code without much thought about the consequences. Take, for instance, the subtle differences in implementations of the modulo operator across programming languages: http://en.wikipedia.org/wiki/Modulo_operator#Common_pitfalls
It would be great if scientific software were open. Unfortunately, it won't matter a lick if it is.
This is only compounded by the fact that reading source code sucks. Source code is an end result of multiple processes that occur in feedback loops. With just the source code, you never see how the code got that way. It's like showing someone a maze with the start and end points marked but the middle of the map blocked out.
Different programmer's conceptions of what constitutes good code varies widely. One man's golden code is another's garbage. Just because the source code is available doesn't mean anyone is going to understand it or be able to work with it effectively.
Compounding this all is the fact that few people are going to want to read the source code. Analyzing source code is dull work, maybe the worst job a programmer can take while still doing programming. Most programmers are far happier to discard old code and start from scratch. This is often a bad idea and doesn't lead to a better product, but at least you don't want to kill yourself while you're doing it.
When it comes to reproducing algorithmic results, I would prefer having a description of the algorithm, a set of inputs, and a set of outputs. I would then write the actual code myself and see if I get the same results. This, I think, is much closer to the concept of reproducing lab results in the physical sciences. You wouldn't use the same exact particle accelerators if you were verifying the results from a paper on nuclear physics. I'm afraid having access to the raw source code will be used as a crutch where logic errors are missed from reusing portions of code without much thought about the consequences. Take, for instance, the subtle differences in implementations of the modulo operator across programming languages: http://en.wikipedia.org/wiki/Modulo_operator#Common_pitfalls
It would be great if scientific software were open. Unfortunately, it won't matter a lick if it is.