No, that's expected. The program you wrote might be minimalistic, but there is still a lot of code under the hood for the runtime.
If you compile a program that's twice as complex, you should still get roughly the same build time.
FWIW The compiler builds in about 30 seconds. And it's a really complex piece of software.