I would say this is also an opposing argument: https://twitter.com/ylecun/status...

YeGoblynQueenne · on Sept 30, 2019

Yes, gradient optimisation always gets the final word. But what does that word, well, say?

                                     _
                                    / \
            ,----------------------------. 
            | Hey! I found your minimum! |
            '------------    ------------'                                    /
                 .----.  \  /   /         \                                  /
                /      \  \/   /           \                                /
               /        \  @  /             \                              /
              /          `'--'               \                            /
             /                                \                          /
            /                                  \                        /
           /                                    \                      /
          /                                      \   .-.              /
         /                                        \_/   `\           /
   \.__.'                                                 \         /
                                                           \       /
                                                            `;._.-'