While the majority of existing homebrew are either games or emulators, the systems are not limited to just playing games and there are a few options for programming them. devkitPro[0] provides a cross-compiling toolchain for 3ds homebrew development, if you are comfortable with C/C++ it is not all that difficult to program on the platform. There are several "portlibs" including SDL (SDL1 not SDL2 unfortunately) provided by devkitPro, which come in handy if you don't want to learn low level 3ds-specific APIs. There are also some homebrew game engines[1] that allow running Lua-based programs, though I haven't tried these myself.
I have the HP Prime that is basically this. The programming language is pretty terrible but you can get pretty impressive 3D plots (with touchscreen as well!) if you install a plugin