The value proposition of deep learning is to eliminate these kind of hand-coded features and to discover the features automagically.
However, maybe there's a middle ground. I.e. maybe we don't need the more esoteric features that SIFT uses, but it just makes sense to do edge detection, and say a Fourier transform for audio.
The true value proposition of deep learning is not to avoid hand-coded features, but to make better use of scale in data and computational resources.
More specifically, adding SIFT or edge detection to your raw pixel input will almost always strictly improve a deep model's performance (though they might be redundant) at a not-particularly-large computational cost.
It wouldn't solve the adversarial example problem though, except to the extent that it makes calculating gradients harder.
I wrote [1], I'm plenty aware of the "feature discovery" that goes on, but it can still be an enhancer. See my recent talk[2] for a good overall idea of the situation.
Normalization and other data transforms are still required for discovery of features.
That library looks like garbage. The website describes convnets as a "type of restricted Boltzmann machine." How can anyone trust a library with that level of misinformation?
I appreciate the feedback. I need to clarify the implementation in DL4j in the documentation.
The impl is a convolutional RBM. That being said, where's your deep learning library? ;)
The documentation is still being worked on. If that's the only thing you can cherry pick from a new project, I must not be doing too bad.
Edit: Clarified. I know you're just a troll account, but I'll throw this out there anyways, I think anyone who judges the quality of a lib based on a 2 second reading isn't qualified to judge much. A lot of it does rely on me with the documentation, but I'd love to chat with you one on one if you think I'm not qualified. Trash me all day, I'll either learn something or embarrass you. Both are fine with me ;).
A lot of people like Torch. I haven't looked at pylearn2 in a while, but that might be good too. Then there are a few researcher libraries with limited documentation. Nitish Srivastava has his DeepNet library and George Dahl has gdbn and there are certainly a few others too. Who knows, maybe people will start contributing documentation.
I would probably recommend Torch at this point. The incentives don't exist for the experts to make really good open source projects and spend all the time required maintaining them and helping people with them.
That's fine. I do this full time. Despite it being new, I'm coming at it from a stand point of providing a platform for newer users and apps around it. You would be surprised the demand for industry.
You're right about this which is why I started a company around it.
I've already talked with andrew ng and yoshua bengio. My incentives are different from there's, however, I do have their blessings to continue doing this.
I walked in to this expecting skeptics. That being said, I love deep learning as a field and will be implementing every possible neural net I can. Since my incentives are different, I can explore the different use cases with customers and help further the field in different directions that might not make sense for say baidu, facebook ,or google.
The value proposition of deep learning is to eliminate these kind of hand-coded features and to discover the features automagically.
Yes and just to clarify - this seems like an indication that however useful, deep learning can't follow through with that promise. And I think the ability to do this the key thing - all the approaches work at some level but without this "automagicity", each become hostage to brow-wrinkling experts who become the only one to understand the black-magic of algorithm tuning.
However, maybe there's a middle ground. I.e. maybe we don't need the more esoteric features that SIFT uses, but it just makes sense to do edge detection, and say a Fourier transform for audio.